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SECTION  1 

INDUSTRY  TRUSTED  COMPUTER  SYSTEM  EVALUATION  PROGRAM 


INTRODUCTION 


PURPOSE 

The  purpose  of  this  document  is  to  propose  a  process  by  which 
trusted  computer  system  developments  may  be  reviewed  and  evaluated 
under  the  DOD  Computer  Security  Initiative  Program.  A  trusted 
computer  system  is  one  which  employs  sufficient  hardware  and 
software  integrity  measures  to  allow  its  use  for  simultaneously 
processing  multiple  levels  of  classified  and/or  sensitive 
information.  This  document  describes  a  process  by  which 
manufacturers  may  submit  their  proposed  products  for  evaluation,  and 
by  which  a  government-wide  evaluation  center  may  conduct  the  review 
and  evaluation.  The  criteria  for  evaluation  of  a  trusted  computer 
system  are  described,  as  well  as  the  proposed  roles  of  the 
manufacturer  and  the  center. 


OVERALL  PROGRAM 

The  Department  of  Defense  Computer  Security  Initiative  was 
established  in  1978  with  the  goal  of  achieving  the  widespread 
availability  of  trusted  computer  systems.  Three  objectives  must  be 
met  to  accomplish  this  goal:  1)  effectively  demonstrate  trusted 
computer  systems  in  a  variety  of  applications;  2)  Involve  the 
commercial  computer  industry  in  the  development  of  trusted  ADP 
systems,  and  3)  establish  mechanisms  for  evaluation  of  trusted  ADP 
systems. 

To  accomplish  the  first  objective,  three  systems  are  being 
developed.  They  are  the  Kernelized  Secure  Operating  System  (KSOS- 
11)  for  the  PDP-11 /70  computer,  the  Kernel Ized  Virtual  Machine  (KVM) 
for  the  IBM-370  series  computer,  and  KSOS-6  for  a  modified  Honeywell 
level  6/43  mini -computer. 

The  second  objective  is  being  accomplished  through  the 
educational  aspect  of  the  initiative.  Seminars  and  forums  are  an 
essential  part  of  this  program  in  order  to  make  available  research 
results  and  user  requirements  to  stimulate  computer  manufacturers  to 
develop  new  systems  which  will  be  suitable  for  trusted  use  by  the 
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DoD  «nd  other  government  agencies. 

Several  elements  of  the  third  objective  have  already  been 
considered.  Nlbaldl  has  proposed  a  set  of  requirements  for  a 
trusted  computing  base  C5D,  and  evaluation  criteria  CO.  Central  to 
the  Implementation  of  an  approval  process  Is  the  concept  of  a  Dob  or 
government  evaluation  center.  This  evaluation  center  would  perform 
technical  "laboratory  evaluations"  of  commercial  systems  to  be  used, 
or  suitable  for  use.  In  federal  applications  requiring  a  trusted 
system.  For  each  system  evaluated,  the  center  will  produce 
documentation  categorizing  the  system  end  describing  Its  specific 
protection  attributes. 


OVERVIEW 

The  goal  ov  the  review  and  evaluation  process  Is  to  determine 
the  protection  provided  by  a  computer  system  by  comparing  the 
features  of  the  system  to  those  features  required  for  a  specific 
level  of  protection  as  defined  by  Nlbaldl  C43. 

The  evaluation  process  will  begin  when  a  commercial  computer 
manufacturer  or  a  government  agency  requests  evaluation  of  a 
computer  system.  Before  a  complete  evaluation  Is  started,  the 
center  will  determine  the  potential  the  system  has  for  use  in  a 
trusted  application  by  examining  the  system  documentation.  If  It  Is 
determined  that  the  system  Is  suitable,  a  full  evaluation  will  be 
performed  by  the  center  to  determine  the  level  of  protection 
provided  by  the  system  and  ultimately  to  place  the  product  on  an 
Evaluated  Products  List  (EPL). 

It  Is  of  value  to  both  the  DoD  and  the  manufacturer  to  begin 
the  evaluation  process  early  In  the  product  development  cycle  In 
order  to  have  the  most  beneficial  effect  on  the  protection  designed 
into  the  system.  To  provide  a  context  for  the  description  of  an 
evaluation  process,  section  2  defines  the  product  development  cycle 
for  government  and  industry.  Section  3  contains  a  description  of 
the  evaluation  criteria  and  provides  a  basis  for  how  the  system 
evaluation  levels  are  used.  Finally,  section  4  defines  the  process 
that  will  be  used  by  the  Security  Evaluation  Center  to  evaluate 
computer  systems  produced  as  trusted  commercial  products. 


SECTION  2 


DEVELOPMENT  CYCLE  FOR  PRODUCTS 


The  development  of  computer  systems  Is  •  complex  process 
containing  a  series  of  phases.  Biggs,  et.  at  C2J  provides  a 
comprehensive  description  of  a  system  development  process  which 
proceeds  through  four  phases.  The  development  process  for  DoD- 
contracted  products  is  similar  to  that  described  by  Biggs.  One 
major  difference  is  that  the  DoD  development  cycle  is  characterized 
by  a  series  of  well-defined  specifications.  First,  a  system 
specification  which  is  called  an  A-specif 1  cat  ion,  second,  a 
development  specification  which  is  a  part  I  or  B-specification,  and 
third,  a  product  specification  which  is  a  part  II  or  C- 
specif ication.  A  second  major  difference  between  the  two 
development  processes  is  in  the  area  of  design  reviews  and  audits. 
For  a  government  or  DoO-contracted  product  a  series  of  design 
reviews  is  required  as  the  product  progresses  through  the  cycle. 
These  reviews  are:  1)  system  requirements  review,  2>  system  design 
review,  3)  preliminary  design  review,  4)  critical  design  review,  5) 
functional  configuration  audit,  and  6)  physical  configuration  audit. 
For  a  commercial  product,  the  design  reviews  and  audits  are  internal 
to  the  developers.  In  general,  the  design  team  will  provide  design 
reviews  to  a  steering  committee  at  well-specified  milestones  In  the 
development  cycle. 

For  purposes  of  clarity  in  this  paper  we  define  the  DoD-product 
development  cycle  '  as  containing  four  phases  which  are:  1)  concept 
formulation,  2)  validation,  3)  full-scale  engineering  development, 
and  4)  production.  These  phases  are  similar  to  those  described  by 
Biggs  which  are:  1)  systems  planning,  2)  systems  requirements,  3) 
systems  development,  and  4)  systems  implementation.  The  following 
paragraphs  define  the  DoD  development  cycle  in  terms  of  activities 
that  might  take  place  in  the  course  of  commercial  system 
development. 


Other  government  agencies  have  similar  acquisition  procedures, 
however,  the  DoD  procedure  is  referenced  here  because  it 
Incorporates  all  cf  the  critical  aspects  of  procurement  and  Is  the 
one  most  familiar  to  the  authors. 
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CONCEPT  FORMULATION  PHASE 


The  concc;.t  formulation  phase  la  an  embryonic  state  where  Ideas 
are  brought  together  to  create  a  new  system,  or  to  modify  or  enhance 
an  existing  one.  Ideas  may  originate  In  any  part  of  an  organization 
but  most  often  are  generated  within  marketing  or  engineering. 

Further  refinement  and  formulation  of  new  systems  are  achieved 
through  the  Interaction  of  management,  marketing,  engineering,  and 
manufacturing  until  the  concept  Is  complete,  and  a  commitment  Is 
made  to  develop  the  system.  Prior  to  making  a  commitment,  a 
feasibility  study  may  be  done.  Throughout  the  early  stages  of 
concept  formulation,  the  manufacturer  will  be  dealing  with 
engineering  or  mcrketlng  notes.  Informal  papers,  partial 
documentation,  and  often,  sensitive  marketing  plans  and  Information. 
Ideas  for  new  products  are  rigorously  protected  from  competition 
throuqhout  development,  but  are  particularly  sensitive  In  this  early 
stage. 

Once  a  commitment  has  been  made  by  management  to  develop  the 
system,  the  engineering  organisation  will  begin  to  prepare  a  system 
level  specification  for  the  product.  This  Is  equivalent  to  the  Dob 
A-level  specification  defining  the  general  nature  of  the  system. 
Including  functional  and  performance  requirements,  and  the 
Interfaces  with  existing  products.  For  a  Dob  or  government 
contracted  system,  a  System  Requirements  Review  (SRR)  will  be  held 
to  determine  the  Initial  direction  and  progress  of  the  contractor's 
system  engineering  effort. 


VALIDATION  PHASE 

This  phase  involves  an  Internal  review  cycle  with  the 
engineering  organisation  responsible  for  the  product  design.  The 
development  specification  Is  usually  produced  during  this  phase. 

This  specification  describes  the  design  of  the  system.  Including 
allocation  of  functional  performance  requirements  to  modules,  and 
the  tests  required  to  determine  compliance  of  the  modules  to  the 
specification.  The  development  specification  Is  a  complete 
statement  of  the  performance,  design.  Interface,  and  formal 
qualification  test  requirements. 

If  the  system  is  to  be  verified  to  a  security  modal,  a  formal 
Top-Level  Specification  must  be  generated  as  part  of  the  development 
specification.  This  specification  defines  all  functions  visible  at 
the  user  Interface  In  a  mathematically  unambiguous,  non-procedural, 
verifiable  language.  The  verification  should  demonstrate  that  the 
design  described  by  the  top-level  specification  does  not  violate  the 
rules  of  a  security  model. 


This  phase  ends  with  a  System  Design  Review  (SDR)  which  Is  a 
final  check  on  the  system  specification  to  ensure  the  completed 
specification  adequately  specifies  the  system  requirements. 


FULL-SCALE  DEVELOPMENT  PHASE 

In  this  phase,  preliminary  and  detailed  design  of  the  system 
take  place.  In  the  development  of  a  DoD  contracted  Item,  there  are 
at  least  two  design  reviews;  preliminary  and  critical.  A 
Preliminary  Design  Review  (PDR)  Is  held  after  authentication  of  the 
development  specification  and  completion  of  the  preliminary  design. 
This  review  Is  a  formal  technical  review  of  the  basic  design 
approach.  The  Critical  Design  Review  (CDR)  Is  held  after  a  draft 
product  specification  Is  produced.  This  specification  defines  how 
to  actually  build  the  system  by  specifying  the  exact  product 
configuration  and  detailed  technical  description.  The  purpose  of 
the  CDR  Is  to  review  the  draft  product  specification  and  to  ensure 
that  It  satisfies  the  performance  and  functional  requirements 
established  by  the  development  specification.  This  review  Is  a 
formal  technical  review  of  the  detailed  design  to  establish  the 
integrity  of  the  design  prior  to  code  and  test. 


PRODUCTION  PHASE 

The  production  phase  Includes  the  actual  manufacture  of 
hardware  Items  and  the  generation  of  software  to  complete  the 
computer  system.  All  necessary  testing  is  performed  to  ensure  that 
the  system  meets  the  technical,  functional,  and  performance 
requirements  of  the  specifications.  Field  support  services  and 
documentation  are  developed  In  preparation  for  the  deployment  of  the 
system  to  end  users.  Deployment  Includes  the  actual  sale  arid 
installation  of  the  system  to  the  customer. 

i 

Early  In  this  phase  a  Functional  Configuration  Audit  (FCA)  Is 
helo  to  verify  that  development  has  been  completed  satisfactorily 
and  that  the  actual  system  performance  complies  with  the  development 
specification.  During  this  phase,  the  product  specif Icatlon  Is 
finalized.  The  final  review  of  the  system  Is  the  Physical 
Configuration  Audit  (PCA)  which  verifies  that  the  "as  built"  system 
conforms  to  the  technical  documentation.  The  Integrity  of  the 
product  specification  Is  established  by  the  PCA. 


SECTION  3 


EVALUATION  CRITERIA 


EVALUATION  PREREQUISITES 

An  lapcrtant  requirement  of  the  evaluation  prograa  both  froa 
the  viewpoint  of  the  aanufacturer  and  the  governaent  la  that  the 
evaluation  be  consistent  for  all  products.  To  achieve  this,  a 
detailed  set  of  evaluation  criteria  Is  needed  that  will  allow  both 
the  protection  value  of  architectural  features  and  the  assurance 
value  of  developaent  techniques  to  be  well  defined.  In  addition.  It 
Is  necessary  that  the  criteria  be  Independent  of  architecture  so 
that  Innovation  Is  not  lapeded. 

The  proposed  evaluation  criteria  and  process  are  both  generic 
and  specific:  evaluation  factors  have  been  defined,  and  various 
degrees  of  rigor  for  each  factor  have  been  Incorporated  Into  seven 
hierarchical  protection  levels  representing  both  systea-vide 
protection  and  assurance  that  the  protection  Is  properly 
lapleaented. 


EVALUATION  FACTORS 

There  are  three  "prise  evaluation  factors"  and  nuaerous 
subsidiary  factors  described  by  Nlbaldl  lO.  These  are  listed  In 
figure  1.  The  factors  are  Independent  of  the  architecture  of  the 
trusted  systea  to  be  reviewed  and  evaluated,  except  for  the 
"prevention"  factur  of  "aechanlsa"  which  aay  be  architecturally 
dependent.  The  basis  for  evaluation  of  the  prevention  portion  of 
the  protection  aechanlsa  Is  a  Trusted  Coaputlng  Base  (TCB)  (the 
equivalent  of  a  security  kernel  and  non-kernel  security  related 
software).  A  specification  for  a  TCB  Is  given  by  Nlbaldl  C5D.  This 
specification  describes  the  prlaltive  operating  systea  and 
and lllary  "trusted  processes"  that  are  required  for  a  trusted 
systea.  The  specification  Is  generic  In  that  it  Is  applicable  to 
the  several  different  protection  bases  understood  today  (e.g. 
descriptor-based  and  capability-based  systeas).  Future 
technological  advances  resulting  In  new  systea  architectures  aay 
require  aodi fleet  ion  of  the  description  of  a  TCB,  but  trusted 
systeas  currently  under  developaent  can  be  evaluated  using  the 
present  TCB. 

It  can  be  ».;ipected  that  the  detail  available  In  the 
descriptions  of  the  factors  will  Increase  and  eature  as  the 
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evaluation  process  is  applied  to  trusted  systems  under  development. 

Figure  1.  EVALUATION  FACTORS 
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PROTECTION  LEVELS 

The  seven  levels  of  protection  proposed  by  Nibaldi  [43  are 
described  in  figure  2.  When  a  system  is  evaluated  it  will  receive  a 
rating  determined  by  the  highest  protection  level  that  is  completely 
satisfied.  The  seven  levels  are  cumulative  in  that  a  rating  at  a 
certain  level  requires  that  the  criteria  at  that  level  and  all  lower 
levels  be  satisfied.  Thus  a  system  that  has  satisfied  all  of  the 
requirements  except  one  for  a  "Level  3"  will  be  assigned  a  "Level 
2".  The  results  of  applying  the  process  will  be  to  develop  a  list 
of  products  that  have  undergone  evaluation,  and  thus  are  eligible 
for  use  in  applications  requiring  a  trusted  system.  This  list  of 
systems  has  been  designated  an  evaluated  products  list  (EPL). 


LEVEL  0:  NO  PROTECTION 

IF  THERE  IS  NO  INDICATION  WITHIN  THE  THREE  AREAS  THAT  A  SYS¬ 
TEM  CAN  PROTECT  INFORMATION,  THE  SYSTEM  RECEIVES  A  LEVEL  0 
EVALUATION. 

LEVEL  1 :  LIMITED  CONTROLLED  SHARING 

LEVEL  1  APPLIES  TO  SYSTEMS  WHICH  HAVE  DATA  ACCESS  CONTROLS 
CAPABLE  OF  PROVIDING  ONLY  LIMITED  PROTECTION. 

LEVEL  2:  EXTENSIVE  MANDATORY  SECURITY 

THE  SYSTEM  PROTECTION  PROVIDES:  1)  ADMINISTRATIVELY  CON¬ 
TROLLED  AUTHORIZATION  TO  READ  DATA,  2)  FLOW  CONTROL  TO  PREVENT 
DATA  COMPROMISE,  AND  3)  WRITE  ACCESS  CONTROL. 

LEVEL  3:  STRUCTURED  PROTECTION  MECHANISM 

THE  PROTECTION  MECHANISMS  MUST  BE  CLEARLY  IDENTIFIED, 
ISOLATED  AND  MADE  INDEPENDENT  OF  OTHER  SOFTWARE.  TRUST  IS 
GAINED  THROUGH  METHODOLOGICAL  DESIGN  OF  THE  PROTECTION- 
RELATED  COMPONENTS  OF  THE  OPERATING  SYSTEM  (I.E.,  THE  TCB)  AND 
MODERN  PROGRAMMING  TECHNIQUES.  ADEQUATE  TEST  RESULTS  ARE 
STILL  THE  PRIMARY  MEANS  OF  ASSURANCE. 

LEVEL  4:  DESIGN  CORRESPONDENCE 

AT  THIS  LEVEL  FORMAL  METHODS  ARE  EMPLOYED  TO  CONFIRM 
TRUSTWORTHINESS  OF  THE  DESIGN.  MATHEMATICAL  PROOFS  OF  COR¬ 
RESPONDENCE  OF  THE  DESIGN  TO  A  SECURITY  MODEL  ARE  REQUIRED. 

LEVEL  5:  IMPLEMENTATION  CORRESPONDENCE 

THE  SYSTEM  MUST  BE  SHOWN  TO  CORRESPOND  TO  THE  VERIFIED 
TOP-LEVEL  DESIGN.  MORE  STRINGENT  REQUIREMENTS  FOR  DENIAL  OF 
SERVICE,  HARDWARE  FAULT  TOLERANCE,  AND  LEAKAGE  CHANNEL  CON¬ 
TROL  ARE  DEMANDED. 

LEVEL  6:  OBJECT  CODE  ANALYSIS 

A  FORMAL  ANALYSIS  OF  THE  OBJECT  COOE  IS  REQUIRED  TO  PROVE 
THAT  THE  IMPLEMENTATION  SOFTWARE  FULFILLS  THE  REQUIREMENTS  OF 
THE  SECURITY  MODEL.  FORMAL  METHODS  OF  VERIFICATION  MUST  ALSO 
BE  APPLIED  TO  THE  HARDWARE. 


Figure  2.  TRUSTED  SYSTEM  PROTECTION  LEVELS 


APPLICATION  OF  PROTECTION  LEVELS 


It  Is  worth  commenting  on  how  the  evaluation  levels  will  be 
used  by  those  In  the  government  outside  of  the  evaluation  center. 

In  order  for  the  protection  to  be  applied  In  a  uniform  way  across 
many  systems  being  procured/  there  must  be  a  relatively  small  set  of 
numbers  describing  protection  that  can  be  mapped  to  another 
relatively  small  set  of  numbers  characterizing  the  environment  Into 
which  the  particular  system  Is  to  be  Installed.  The  levels 
structure  described  above  satisfies  the  requirements  for  simplicity 
and  consistency.  Similarly  simplifying  descriptive  factors  have 
been  developed  for  the  environmental  risk  presented  by  an 
installation  or  an  application  (in  the  context  of  the  DoD 
classification  system)  by  Adams  C13.  Some  exploration  of  a  mapping 
between  environmental  factors  and  protection  levels  was  performed 
during  the  Air  Force  Computer  Security  Summer  Study  [33  and  the 
results  suggest  that  this  will  be  a  viable  way  to  allow  various 
federal  agencies  to  define  their  risks  and  to  guide  procurement 
offices  in  the  specification  of  the  underlying  protection  required 
for  their  agencies'  systems. 

There  is  a  concern  that  reducing  everything  in  situations  as 
complex  as  these  evaluations  to  a  small  set  of  numbers  will  result 
in  a  rigid  and  unthinking  application  of  the  numbers  without 
consideration  of  the  specific  protection  offered  by  a  trusted 
system.  While  this  is  certainly  possible/  It  Is  more  likely  that 
operational  requirements  will  keep  this  from  happening  and  result  In 
the  numbers  being  used  as  guldeposts  for  both  the  acquisition  agency 
and  the  certification  authority.  Thus/  we  would  expect  a  procuring 
agency  to  determine  that  the  protection  offered  by  a  "level  4" 
system  is  required  for  a  particular  system  being  acquired  and  to 
include  such  a  statement  in  the  Request  for  Proposal  (RFP). 
Contractors  responding  to  the  RFP  would  be  expected  to  describe 
their  proposed  system  designs  and  how  they  would  make  proper  use  of 
the  protection  features  of  the  trusted  system  they  have  chosen  as 
the  basis  for  their  proposal.  During  source  selection/  the  various 
bidders  would  be  evaluated  on  the  basis  of  how  well  they  structured 
their  application  design  to  take  advantage  of  the  strong  points  In 
the  architecture  of  the  underlying  trusted  system  and  to  minimize 
the  weak  points. 

It  would  be  possible  for  a  bidder  to  propose  •  "Itvel  3” 
system/  but  It  would  be  Incumbent  on  him  to  convince  the  source 
selection  team  that  he  has  built  his  system  on  the  strong  features 
of  the  particular  "level  3"  system  and  has  appropriately  addressed 
the  deficient  areas.  This  allows  the  bidders  to  have  some 
flexibility  In  their  choice  of  systems;  commensurate  with  the  fact 
that  they  and  the  procuring  agency  will  eventuelly  heve  to 
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demonstrate  to  the  certification  authority  that  the  proposed  system 
satisfies  the  specified  requirements,  fhls  also  addresses  the 
concern  about  a  “level  3"  system  being  "almost  a  level  4  system 
except  for...". 
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SECTION  4 


TRUSTED  COMPUTER  SYSTEM  EVALUATION  PROCESS 


OVERVIEW  OF  THE  PROCESS 
Steps  of  the  Evaluation  Process 

The  system  evaluation  process  described  herein  consists  of  four 
sequential  steps:  1)  preliminary  evaluation,  2)  interactive 
evaluation,  3)  final  evaluation,  and  4)  periodic  re-evaluation.  The 
preliminary  evaluation  step  is  a  determination  of  the  suitability  of 
an  industry  developed  system  for  evaluation  based  upon  the  design  of 
the  TCB  of  the  system.  When  the  TCB  has  been  adequately  specified, 
the  system  will  be  ready  for  an  interactive  evaluation.  The 
interactive  evaluation  is  a  review  of  the  system  design  in  terms  of 
the  TCB  and  the  means  by  which  the  system  satisfies  the  criteria  for 
the  level  of  protection  which  the  manufacturer  specifies.  The  final 
evaluation  involves  analysis  and  testing  of  the  completed  system  to 
determine  the  level  of  protection  provided  and  the  strengths  and 
weaknesses  relative  to  that  level.  Periodic  re-evaluation  applies 
to  those  systems  that  are  modified  after  a  final  evaluation  has  been 
completed. 

Relationship  Between  the  Evaluation  Process  and  the  Development  Cycle 

A  graphic  representation  of  the  relationship  between  the 
evaluation  process  and  the  product  development  cycle  is  shown  in 
figure  3.  The  evaluation  process  is  shown  as  a  set  of  four 
sequential  steps,  while  the  product  development  cycle  is  shown  as 
four  phases.  The  arrows  indicate  the  required  sequence  for  the 
evaluation  of  a  system.  The  "request  for  evaluation"  may  be 
initiated  during  any  of  the  four  phases  of  development,  but  because 
the  evaluation  process  is  independent  of  the  development  cycle  it 
will  always  consist  of  the  four  steps  in  sequence.  A  relative  time 
line  is  shown  to  indicate  that:  1)  the  evaluation  may  begin  during 
the  concept  formulation  phase,  2)  the  interactive  evaluation  will 
end  when  all  specifications  (system,  development,  and  product)  are 
complete,  and  3)  the  final  evaluation  will  take  place  when  the 
system  is  available. 
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PHASES  OF  DEVELOPMENT  SECURITY  EVALUATION 


Figure  3.  EVALUATION  PROCEDURE 


PRELIMINARY  EVALUATION 


Preliminary  evaluation  is  an  analysis  of  the  TCB  of  a 
manufacturer's  system  to  determine  the  adequacy  of  that  system  for 
use  in  an  environment  requiring  trusted  access  controls.  The 
purpose  of  this  evaluation  is  to  determine  whether  or  not  the 
manufacturer's  system  is  sufficiently  designed  and  documented,  in 
terms  of  the  TCB  and  the  evaluation  criteria,  to  begin  an 
interactive  evaluation.  When  the  manufacturer  requests  an 
evaluation,  he  will  provide  the  evaluation  center  with  complete 
system  documentation  and  indicate  the  target  level  of  protection  he 
hopes  to  achieve.  The  documentation  should  describe  the  computer 
system  under  development  in  terms  of  the  TCB  specification,  and 
detail  the  design  and  implementation  of  the  system  in  terms  of  the 
technical  evaluation  criteria.  The  preliminary  evaluation  will 
determine  if  the  TCB  can  possibly  provide  this  "target"  level  of 
protection  by  analysis  of  the  design  methodology  and  the  hardware 
and  software  mechanisms  provided  by  the  system. 

Although  It  is  presumed  that  most  evaluations  will  be  conducted 
on  systems  that  have  been  designed  with  verifiable  protection  in 
mind  from  the  onset,  there  may  be  released  (production)  systems  that 
have  a  sufficient  protection  base  to  allow  restructuring  of  the 
system  to  incorporate  a  TCB.  In  this  case,  the  focus  of  the 
preliminary  evaluation  will  be  on  the  modifications  necessary  to  the 
production  system  (in  structure,  documentation  and  testing)  to 
satisfy  the  criteria  for  one  of  the  protection  levels. 

In  a  preliminary  evaluation,  the  developer  will  define  the 
suitability  of  his  system  for  use  in  an  environment  requiring  a 
trusted  computer  system.  His  presentation  will  cover  the  areas  of 
hardware  architecture,  proposed  design  and  development  methodology, 
verification  methodology,  if  any,  and  software  architecture.  The 
manufacturer  is  responsible  for  presenting  the  system  in  a  top-down 
fashion,  and  for  focusing  on  the  ways  in  which  the  system  embodies 
the  TCB  and  the  system  design  satisfies  the  evaluation  criteria  for 
the  target  level  of  protection  specified.  The  necessary  mechanism 
and  assurance  provided  by  the  system  for  the  specified  target  level 
must  be  outlined. 

When  the  security  evaluation  center  receives  a  request  to 
evaluate  a  system,  a  team  will  be  formed  to  perform  the  evaluation. 
The  output  of  the  preliminary  evaluation  will  be  the  team's 
assessment  of  the  status  of  the  system  and  the  potential  the  system 
has  of  achieving  the  level  of  protection  stated  by  the  manufacturer 
or  the  highest  level  the  system  might  achieve  based  on  the 
information  available.  The  assessment  may  indicate  that  the  system 
is  not  yet  ready  to  proceed  to  a  full  interactive  evaluation.  This 
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would  occur  if  the  specification  has  not  been  well  defined  in  teres 
of  the  TCB,  or  if  the  complexity  or  method  of  Implementing  the  TCB 
is  not  amenable  to  this  type  of  evaluation.  In  that  case,  the 
evaluation  team  will  identify  what  further  information  is  needed,  or 
what  steps  should  be  taken  before  the  system  is  ready  for 
interactive  evaluation. 


INTERACTIVE  EVALUATION 

The  interactive  evaluation  is  a  logical  extension  of  the 
preliminary  evaluation,  which  will  begin  when  a  preliminary 
evaluation  indicates  the  product  is  suitable  as  a  trusted  system. 

The  review  of  the  system  will  focus  cn  the  TCB,  while  the  review  of 
the  system  design  will  focus  on  the  evaluation  criteria  (i.e.  how 
the  design  satisfies  the  criteria  for  the  level  of  protection 
specified  in  the  preliminary  evaluation).  The  method  of  conducting 
the  evaluation  will  be  a  series  of  presentations  given  by  the 
developer,  together  with  documentation  appropriate  to  the  level  of 
development  of  the  system.  The  government  design  review  process 
which  has  been  used  for  the  KSOS-6  and  KSOS-11  developments  is  the 
model  for  this  interaction.  The  areas  of  hardware  and  software 
which  were  covered  in  the  preliminary  evaluation  will  now  be  covered 
in  depth  by  the  manufacturer's  design  team. 

If  the  evaluation  has  been  initiated  prior  to,  or  during,  the 
full-scale  development  phase,  it  will  occur  concurrently  with  the 
development  of  the  system.  If  the  evaluation  process  is  initiated 
later,  in  anticipation  of  subsequent  releases  of  "trusted  versions” 
of  the  system,  any  interactive  evaluation  step  would  take  place 
during  the  manufacturer's  formulation  of  the  releases. 

The  role  of  the  manufacturer  in  the  interactive  evaluation  is 
to  provide  presentations  and  documentation  on  the  system  to  the 
evaluation  center.  As  in  the  preliminary  evaluation,  the  focus  of 
the  presentations  will  be  the  design  of  the  TCB  and  the  satisfaction 
of  the  policy,  mechanism,  and  assurance  factors  of  the  technical 
evaluation  criteria  for  the  manufacturer's  target  level  of 
protection.  The  manufacturer  will  also  determine  the  schedule  for 
presentations  based  upon  his  progress  in  developing  the  system.  One 
possible  method  Is  to  tie  the  presentation  schedule  to  the 
manufacturer's  Internal  design  review  cycle.  Following  each 
internal  review,  the  manufacturer  could  present  a  similar  review  for 
the  evaluation  team,  but  with  emphasis  on  the  TCB  and  the  evaluation 
criteria.  In  this  way,  the  evaluation  team  will  be  aware  of  the 
direction,  methods,  and  conclusions  pertinent  to  the  system  design, 
without  interfering  with  the  manufacturer's  internal  design  review 
cycle,  and  the  manufacturer  will  be  aware  of  his  progress  relative 
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to  the  target  level  of  protection. 

The  role  of  the  evaluation  tea*  is  to  review  the  system  design 
as  presented  by  the  designers  and  to  point  out  security  relevant 
design  tradeoffs  the  vendor  may  have  overlooked.  The  issue 
addressed  is  the  compromise  of  the  system  through  data  security  and 
integrity  flaws,  timing  and  storage  channels,  and  denial  of  service. 
The  evaluation  team  will  not  attempt  to  re-design  the  manufacturer's 
system  in  any  way,  rather,  it  has  the  responsibility  to  point  out 
flaws  or  potential  flaws  to  the  system  designers.  Since  the 
development  of  a  computer  system  can  extend  over  years,  the 
evaluation  team  will  provide  the  manufacturer  with  in-progress 
reports  detailing  the  progress  or  current  status  of  the  system 
relative  to  the  evaluation  criteria  for  the  target  level,  that  is, 
the  teams  assessment  of  the  TCB  design  Issues,  and  feedback  on  the 
protection  provided  by  the  system.  The  evaluation  team  will  not 
require  the  manufacturer  to  supply  special  documentation  defining 
the  TCB  provided  the  Internal  documentation  adequately  defines  the 
system  design.  KSOS-6  and  KSOS-11  specifications  prc/ide  examples 
of  the  type  of  specifications  required  for  adequate  system 
definition. 

The  Interactive  evaluation  of  an  industry  system  will  be 
complete  when  the  analysis  of  all  specifications  is  complete.  The 
computer  system  will  then  be  ready  for  final  evaluation. 


FINAL  EVALUATION 

The  final  evaluation  consists  of  analysis  and  testing  of  the 
production  system  to  determine  its  strengths  and  weaknesses  relative 
to  the  criteria  for  a  specific  level  of  protection.  The  developers 
will  provide  the  evaluation  center  with  a  production  system,  or 
suitable  access  to  one,  and  will  provide  details  on  the  test  methods 
and  procedures  used  to  determine  the  way  in  which  the  criteria  have 
been  satisfied  for  the  specified  level  of  protection.  In  addition, 
the  manufacturer  must  show  the  way  In  which  the  test  procedures  map 
to  the  Development  Specification,  or  to  the  Top-Level  Specification 
for  systems  requiring  verification. 

The  final  evaluation  cannot  take  place  until  the  manufacturer 
has  completed  his  internal  acceptance  testing  and  the  system  is 
available  for  field  testing,  so  that  the  evaluation  team  will  have 
complete  access  to  the  system  for  hands-on  testing.  There  is  no 
requirement  that  the  evaluation  occur  as  soon  as  the  system  is 
available.  The  manufacturer  may  choose  to  wait  for  some  future 
release  of  the  system  before  the  final  evaluation  takes  place. 
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The  role  of  the  manufacturer  Is  to  perform  the  actual  detailed 
testing  and  where  necessar,,  verification,  to  clearly  demonstrate 
the  protection  capabilities  of  the  system.  Also,  to  aid  the 
evaluation  team's  analysis  of  the  testing,  the  manufacturer  should 
provide  the  complete  test  plan  and  any  test  data  requested  by  the 
evaluation  center. 

The  evaluation  team  will  determine  what  further  testing  Is 
necessary,  if  any,  to  assure  that  the  system  provides  the  security 
and  integrity  for  the  specified  target  level,  using  the 
manufacturer's  qualification  testing  as  a  starting  point.  The 
result  of  the  final  evaluation  will  be  to  determine  the  "actual" 
level  of  protection  and  to  place  the  system  In  the  evaluated 
products  list.  The  output  from  the  final  evaluation  will  be  In 
three  parts:  1)a  public  document  giving  the  level  of  the  system 
and  the  possible  environments  wuere  it  Is  usable,  2)  a  classified 
flaw  analysis  of  the  system.  Including  limitations  and 
vulnerabilities,  and  where  and  how  the  system  can  be  used,  and  3) 
evaluation  team  notes. 


PERIODIC  RE-EVALUATION 

Computer  systems,  being  dynamic,  will  be  modified  or  enhanced 
at  random  intervals  and  thus  will  require  re-evaluation.  The 
evaluation  center  and  *.-nufi-cturer  will  jointly  analyze  all  system 
changes  to  determine  the  securitv-related  aspects  and  tiius  the 
extent  of  the  re-^valuat ion  needed.  The  higher  the  level  of  the 
system,  the  more  detailed  the  re-evaluation  will  be.  For  example, 
code  related  changes  may  only  effect  systems  of  level  5  or  higher 
where  code  proofs  are  required,  while  design  changes  will  effect 
systems  of  level  *♦  or  higher  since  these  systems  require 
mathematical  pi  oof  of  correspondence  of  the  design  to  a  security 
model. 


TIMING  OF  EVALUATION  REQUEST 

The  evaluation  of  an  industry  developed  computer  system  may 
start  during  any  phase  of  the  product  development  cycle.  As  part  of 
the  evaluation  process,  the  center  hopes  that  its  insight  and 
feedback  to  the  manufacturer  will  tend  to  enhance  the 
trustworthiness  of  the  final  system.  Because  of  this,  the  earlier 
in  the  cycle  the  evaluation  Is  started,  the  greater  the  protection 
potential  for  the  resulting  system,  since  the  security  design  will 
be  reflected  in  all  specifications,  and  because  there  will  be 
maximum  exposure  between  the  development  team  and  the  evaluation 
center.  In  conflict  with  the  idea  of  early  contact  Is  the  need  for 
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adequate  system  definition  and  the  desire  of  the  organization  to 
minimize  exposure  of  Its  sensitive  marketing  plans.  Ideally,  the 
request  will  occur  within  the  concept  formulation  phase  of  the 
product  development  cycle,  but  prior  to  the  completion  of  the  system 
specification  ("A"  specification).  At  that  time  the  system  design 
should  be  well  defined  In  terms  of  the  TCB. 

It  Is  Important  to  note  that  high-level  design  Information 
which  Is  usually  produced  In  the  early  phases  of  development  may  not 
exist  when  evaluation  Is  started  later  In  the  development  cycle. 
Since  this  Information  Is  essential  to  a  proper  evaluation,  the 
manufacturer  may  find  It  necessary  to  produce  specifications  after- 
the-fact.  This  case  will  occur  only  for  systems  designed  for  a  high 
level  of  protection. 


SECURITY  EVALUATION  CENTER 

A  DoD  or  government- level  evaluation  center  Is  envisioned  to 
cany  out  the  process  described  above.  The  center  will  maintain  a 
staff  experienced  In  security  Issues,  TCB  design,  system  design, 
testing,  penetration,  and  Interaction.  In  addition  to  the 
evaluation  of  industry  trusted  systems,  the  staff  will  be  available 
to  government  agencies  requiring  design  or  consultation  on 
individual  products  or  contracts,  especially  In  the  area  of  design 
of  applications  for  trusted  systems.  The  center  will  establish  and 
maintain  an  internal  research  and  development  capability  to  enhance 
and  create  new  development  tools  essential  to  the  system  evaluation 
process.  Among  those  needed  are  techniques  for  final  dynamic 
testing  of  code  to  show  that  required  functions  are  performed 
properly  and  no  unwanted  functions  are  present.  Also  needed  are 
automated  tools  to  transform  system  specifications  into  drivers  for 
final  testing  and  tools  to  aid  manual  analysis  of  specifications. 

In  the  verification  area,  automated  tools  are  needed  to  show 
correspondence  proofs  of  source  code  to  design,  and  object  code  to 
source  code.  A  systematic,  automated  technique  for  penetration 
analysis  is  also  needed.  An  area  c&  research  is  symbolic  code 
testing  in  which  the  execution  of  program  paths  is  "simulated" 
through  a  combination  of  path  analysis  and  program  interpretation. 

The  evaluation  center  will  be  particularly  sensitive  to  the 
issue  of  disclosure  of  the  manufacturer's  information.  To  prevent 
such  disclosure,  the  manufacturer's  documentation  will  be  handled  as 
sensitive  or  proprietary  information. 

The  documentation  produced  by  the  canter  may  be  used  by 
government  agencies  and  by  system  contractors  as  input  into  their 
design  and  acquisition  process  and  by  accreditation  authorities  as 
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Input  to  the  accreditation  process.  The  Doh  procedures  and 
authority  for  accreditation  of  a  specific  Installation  for 
classified  processing  remain  unchanged.  The  Interpretation  of  the 
protection  features  Into  specific  threat  and  application 
environments  oil!  be  done  by  the  agency  or  service  responsible  for  a 
particular  installation.  However,  the  resources  of  the  center  will 
be  available  to  those  Involved  In  the  accreditation  process. 


CONFIGURATION  CONTROL 

Each  manufacturer  will  provide  a  physically  secure  facility 
where  a  master  copy  of  the  software  for  the  evaluated  product  will 
be  maintained  (for  products  of  level  4  and  above>.  The  manufacturer 
must  be  able  to  ensure  that  a  copy  can  be  shown  to  be  an  enact 
replica  of  the  master.  For  some  high  level  products,  the 
manufacturer  will  be  required  to  provide  a  secure  machine  facility 
for  development  and  testing  of  the  trusted  system. 

It  Is  Incumbent  upon  the  system  developers  to  present  as 
complete  and  comprehensive  a  description  of  the  security  design  of 
the  system  as  possible,  carefully  addressing  the  Issues  of  policy, 
mechanism,  and  assurance  as  described  In  the  security  metric  C43. 

The  proof  of  the  design,  of  the  existence  of  mechanisms,  and  of  the 
verification  of  the  system  rests  entirely  with  the  developer. 
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